{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ollama\n",
    "\n",
    "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as Llama 2, locally.\n",
    "\n",
    "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. \n",
    "\n",
    "It optimizes setup and configuration details, including GPU usage.\n",
    "\n",
    "For a complete list of supported models and model variants, see the [Ollama model library](https://github.com/jmorganca/ollama#model-library).\n",
    "\n",
    "## Setup\n",
    "\n",
    "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
    "\n",
    "* [Download](https://ollama.ai/download) and install Ollama onto the available supported platforms (including Windows Subsystem for Linux)\n",
    "* Fetch available LLM model via `ollama pull <name-of-model>`\n",
    "    * View a list of available models via the [model library](https://ollama.ai/library)\n",
    "    * e.g., for `Llama-7b`: `ollama pull llama2`\n",
    "* This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.\n",
    "\n",
    "> On Mac, the models will be download to `~/.ollama/models`\n",
    "> \n",
    "> On Linux (or WSL), the models will be stored at `/usr/share/ollama/.ollama/models`\n",
    "\n",
    "* Specify the exact version of the model of interest as such `ollama pull vicuna:13b-v1.5-16k-q4_0` (View the [various tags for the `Vicuna`](https://ollama.ai/library/vicuna/tags) model in this instance)\n",
    "* To view all pulled models, use `ollama list`\n",
    "* To chat directly with a model from the command line, use `ollama run <name-of-model>`\n",
    "* View the [Ollama documentation](https://github.com/jmorganca/ollama) for more commands. Run `ollama help` in the terminal to see available commands too.\n",
    "\n",
    "## Usage\n",
    "\n",
    "You can see a full list of supported parameters on the [API reference page](https://api.python.langchain.com/en/latest/llms/langchain.llms.ollama.Ollama.html).\n",
    "\n",
    "If you are using a LLaMA `chat` model (e.g., `ollama pull llama2:7b-chat`) then you can use the `ChatOllama` interface.\n",
    "\n",
    "This includes [special tokens](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) for system message and user input.\n",
    "\n",
    "## Interacting with Models \n",
    "\n",
    "Here are a few ways to interact with pulled local models\n",
    "\n",
    "#### directly in the terminal:\n",
    "\n",
    "* All of your local models are automatically served on `localhost:11434`\n",
    "* Run `ollama run <name-of-model>` to start interacting via the command line directly\n",
    "\n",
    "### via an API\n",
    "\n",
    "Send an `application/json` request to the API endpoint of Ollama to interact.\n",
    "\n",
    "```bash\n",
    "curl http://localhost:11434/api/generate -d '{\n",
    "  \"model\": \"llama2\",\n",
    "  \"prompt\":\"Why is the sky blue?\"\n",
    "}'\n",
    "```\n",
    "\n",
    "See the Ollama [API documentation](https://github.com/jmorganca/ollama/blob/main/docs/api.md) for all endpoints.\n",
    "\n",
    "#### via LangChain\n",
    "\n",
    "See a typical basic example of using Ollama chat model in your LangChain application."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Sure! Here's a quick one:\\n\\nWhy don't scientists trust atoms?\\nBecause they make up everything!\\n\\nI hope that brought a smile to your face!\""
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.llms import Ollama\n",
    "\n",
    "llm = Ollama(model=\"llama2\")\n",
    "\n",
    "llm.invoke(\"Tell me a joke\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To stream tokens, use the `.stream(...)` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "S\n",
      "ure\n",
      ",\n",
      " here\n",
      "'\n",
      "s\n",
      " one\n",
      ":\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Why\n",
      " don\n",
      "'\n",
      "t\n",
      " scient\n",
      "ists\n",
      " trust\n",
      " atoms\n",
      "?\n",
      "\n",
      "\n",
      "B\n",
      "ecause\n",
      " they\n",
      " make\n",
      " up\n",
      " everything\n",
      "!\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "I\n",
      " hope\n",
      " you\n",
      " found\n",
      " that\n",
      " am\n",
      "using\n",
      "!\n",
      " Do\n",
      " you\n",
      " want\n",
      " to\n",
      " hear\n",
      " another\n",
      " one\n",
      "?\n",
      "\n"
     ]
    }
   ],
   "source": [
    "query = \"Tell me a joke\"\n",
    "\n",
    "for chunks in llm.stream(query):\n",
    "    print(chunks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more about the LangChain Expressive Language and the available methods on an LLM, see the [LCEL Interface](https://python.langchain.com/docs/expression_language/interface)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-modal\n",
    "\n",
    "Ollama has support for multi-modal LLMs, such as [bakllava](https://ollama.ai/library/bakllava) and [llava](https://ollama.ai/library/llava).\n",
    "\n",
    "`ollama pull bakllava`\n",
    "\n",
    "Be sure to update Ollama so that you have the most recent version to support multi-modal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import Ollama\n",
    "\n",
    "bakllava = Ollama(model=\"bakllava\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import base64\n",
    "from io import BytesIO\n",
    "\n",
    "from IPython.display import HTML, display\n",
    "from PIL import Image\n",
    "\n",
    "\n",
    "def convert_to_base64(pil_image):\n",
    "    \"\"\"\n",
    "    Convert PIL images to Base64 encoded strings\n",
    "\n",
    "    :param pil_image: PIL image\n",
    "    :return: Re-sized Base64 string\n",
    "    \"\"\"\n",
    "\n",
    "    buffered = BytesIO()\n",
    "    pil_image.save(buffered, format=\"JPEG\")  # You can change the format if needed\n",
    "    img_str = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n",
    "    return img_str\n",
    "\n",
    "\n",
    "def plt_img_base64(img_base64):\n",
    "    \"\"\"\n",
    "    Display base64 encoded string as image\n",
    "\n",
    "    :param img_base64:  Base64 string\n",
    "    \"\"\"\n",
    "    # Create an HTML img tag with the base64 string as the source\n",
    "    image_html = f'<img src=\"data:image/jpeg;base64,{img_base64}\" />'\n",
    "    # Display the image by rendering the HTML\n",
    "    display(HTML(image_html))\n",
    "\n",
    "\n",
    "file_path = \"../../../static/img/ollama_example_img.jpg\"\n",
    "pil_image = Image.open(file_path)\n",
    "image_b64 = convert_to_base64(pil_image)\n",
    "plt_img_base64(image_b64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'90%'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_with_image_context = bakllava.bind(images=[image_b64])\n",
    "llm_with_image_context.invoke(\"What is the dollar based gross retention rate:\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
